Skip to main content

Policy Improvement

This section outlines how to improve custom content policies using DynamoGuard's feedback, editing, and retraining workflows.

RLHF Guardrail Improvement​

After modifying a policy—whether updating its definition, adding new examples, or applying feedback—you must regenerate data and retrain the policy to activate the changes. To do this, navigate to the training tab and start by clicking Regenerate Training Data. This will augment the existing training dataset to reflect your changes. After data generation is complete, you can again navigate to the training tab and click Train Policy to trigger retraining.

Regeneration

Editing Policies​

To trigger the RLHF guardrail improvement workflow, you can edit a policy’s definition, training data, and provide feedback from monitoring logs.

Editing Policy Definitions​

Policy definitions can be edited from the Policy Management page. Here, edits can be made to the title, description, or associated behaviors. Allowed and disallowed behaviors can also be added and removed. After editing the policy, regenerate the training data to ensure that it aligns with the updated definition.

Policy Editing

Policy Editing

Adding Training Data​

Training data can also directly be added to a policy by clicking the 'Add Data' button in the Training Data tab. Here, data can either be provided one at a time by providing the example and compliance status or by uploading a file containing a larger set of datapoints. Adding training data will also prompt you to regenerate the rest of your training data so that it's better aligned to the uploaded data.

Adding Data

Editing Datapoints​

Existing training data can directly be edited by clicking into the particular datapoint. Here, the datapoint text or compliance status can be edited.

Adding Data

Adding Feedback Data​

After deploying a policy, you can collect feedback on how it performs in the live environment:

  • Navigate to the Monitoring Log to view feedback on datapoints
  • This feedback can be applied to the policy for continuous improvement

Leaving Feedback

Applying Feedback